Image processing device, image processing system and non-transitory computer readable medium storing program

ABSTRACT

An image processing device includes: an acceptance unit that accepts an image information pair composed of image information before color conversion and image information after color conversion; an accuracy output unit that outputs accuracy of a color conversion property from the plural accepted image information pairs; a color conversion property creation unit that creates the color conversion property from the plural accepted image information pairs; and a display control unit that, when the acceptance unit accepts a new image information pair, controls to display, on a display device, at least image information created by color conversion of image information of the new image information pair before color conversion based on the color conversion property created by the color conversion property creation unit from the image information pair that has already been accepted by the acceptance unit and image information of the new image information pair after color conversion.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC § 119 fromJapanese Patent Application No. 2017-187250 filed Sep. 27, 2017.

BACKGROUND Technical Field

The present invention relates to an image processing device, an imageprocessing system and a non-transitory computer readable medium storinga program.

Related Art

In recent years, due to popularization of appliances, such assmartphones or tablets, users taking and/or browsing digital images havebeen increased. At this occasion, photographing environments varyaccording to effects of illumination light or the like, andphotographing subjects also vary. Therefore, after photographing, it issometimes found that a photographed image is not what a user intended;accordingly, colors or the like of the photographed image are adjustedin general.

SUMMARY

According to an aspect of the present invention, an image processingdevice including: an acceptance unit that accepts an image informationpair composed of image information before color conversion and imageinformation after color conversion; an accuracy output unit that outputsaccuracy of a color conversion property from the plural imageinformation pairs accepted by the acceptance unit; a color conversionproperty creation unit that creates the color conversion property from aplural image information pairs accepted by the acceptance unit; and adisplay control unit that, when the acceptance unit accepts a new imageinformation pair, controls to display, on a display device, at leastimage information created by color conversion of image information ofthe new image information pair before color conversion based on thecolor conversion property created by the color conversion propertycreation unit from the image information pair that has already beenaccepted by the acceptance unit and image information of the new imageinformation pair after color conversion.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a diagram showing a configuration example of an imageprocessing system in the exemplary embodiments;

FIG. 2 is a block diagram showing a functional configuration example ofan image processing device and a display device in the exemplaryembodiments;

FIGS. 3A to 3F are diagrams showing examples of a first image obtainedby a first image obtaining unit;

FIGS. 4A and 4B are diagrams showing an example of pairs of imageinformation before color adjustment and image information after coloradjustment;

FIGS. 5A and 5B are diagrams showing methods of obtaining accuracy fromcolor conversion vectors;

FIG. 6 is a diagram showing an example in which display informationcreated by a first accuracy evaluation display unit is displayed on adisplay unit of the display device;

FIG. 7 is a diagram showing a first example in which display informationcreated by a second accuracy evaluation display unit is displayed on thedisplay unit of the display device;

FIG. 8A is a diagram showing a second example in which displayinformation created by the second accuracy evaluation display unit isdisplayed on the display unit of the display device;

FIG. 8B is a diagram showing a third example in which displayinformation created by the second accuracy evaluation display unit isdisplayed on the display unit of the display device;

FIG. 9 is a diagram showing a fourth example in which displayinformation created by the second accuracy evaluation display unit isdisplayed on the display unit of the display device;

FIG. 10 is a diagram showing an example in which a user's evaluationacceptance screen is displayed on the display unit;

FIG. 11 is a diagram showing an example in which an additionalinformation screen is displayed on a display unit;

FIG. 12 is a diagram showing an example of a color conversion model;

FIG. 13 is a block diagram showing a modified example of a functionalconfiguration of the image processing device and the display device inthe exemplary embodiment;

FIG. 14 is a flowchart illustrating operations of the image processingdevice in a first exemplary embodiment;

FIG. 15 is a flowchart illustrating operations of the image processingdevice in a second exemplary embodiment; and

FIG. 16 is a flowchart illustrating operations of the image processingdevice in a third exemplary embodiment.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments according to the present inventionwill be described in detail with reference to attached drawings.

<Description of Entire Image Processing System>

FIG. 1 is a diagram showing a configuration example of an imageprocessing system 1 in the exemplary embodiments.

As shown in the figure, the image processing system 1 of the exemplaryembodiments includes: an image processing device 10 that performs coloradjustment (color conversion) to an original image taken by a camera 40;a display device 20 that displays an image based on display informationoutputted by the image processing device 10; an input device 30 thatallows a user to input various kinds of information items to the imageprocessing device 10; and a camera 40 that takes a photographing subjectS and creates image information to be subjected to color adjustment bythe image processing device 10.

The image processing device 10 is, for example, a so-calledgeneral-purpose personal computer (PC). Then, under the control by an OS(Operating System), the image processing device 10 performs coloradjustment or the like by causing various kinds of application softwareto operate.

The image processing device 10 includes: a CPU (Central Processing Unit)as a computation unit; and a main memory and an HDD (Hard Disk Drive) asa storage unit. Here, the CPU executes various kinds of software, suchas an OS (Operating System, basic software) or application programs(application software). Moreover, the main memory is a storage regionthat stores various kinds of software or data or the like used forexecuting thereof, and the HDD is a storage region that stores inputdata for various kinds of software or output data from various kinds ofsoftware.

Further, the image processing device 10 includes a communicationinterface (hereinafter, referred to as “communication I/F”) 14 forcommunicating with the outside and an input device, such as a keyboardor a mouse.

The display device 20 displays an image on a display screen 20 a. Thedisplay device 20 is composed of a device having a function ofdisplaying an image, such as, for example, a liquid crystal display forPC, a liquid crystal display television or a projector. Consequently,the display method in the display device 20 is not limited to the liquidcrystal display method. Note that, in the example shown in FIG. 1, thedisplay screen 20 a is provided inside the display device 20; however,when, for example, a projector is used as the display device 20, thedisplay screen 20 a is a screen or the like provided outside the displaydevice 20.

The input device 30 is configured with a keyboard, a mouse or the like.The input device 30 is used to start or exit application software forperforming color adjustment, or, when color adjustment is to beperformed, to input instructions for performing color adjustment to theimage processing device 10 by a user.

The camera 40 is an example of a photographing device and includes, forexample, an optical system that converges incident light and an imagesensor that is an imaging unit to detect the light converged by theoptical system.

The optical system is configured with a single lens or by combiningplural lenses. In the optical system, due to a combination of lenses,coating applied to a lens surface, and so forth, various kinds ofaberrations are removed. The image sensor is configured by arrangingimaging elements, such as CCDs (charge coupled devices) or CMOS(complementary metal oxide semiconductor) elements.

The image processing device 10 and the display device 20 are connectedvia a DVI (Digital Visual Interface). Note that, it may be possible toperform connection via an HDMI (a registered trademark, High-DefinitionMultimedia Interface), a DisplayPort, or the like, instead of the DVI.

Moreover, the image processing device 10 and the input device 30 areconnected via, for example, a USB (Universal Serial Bus). Note that, inplace of a USB, connection may be made via an IEEE1394, an RS-232C, orthe like.

Further, in the example shown in the figure, the image processing device10 and the camera 40 are connected by wire, and, for example, connectedvia a USB, an IEEE1394 or an RS-232C. Consequently, image information ofan image taken by the camera 40 is sent to the image processing device10 by wire. However, the connection is not limited thereto, and it maybe possible to adopt wireless connection, such as using wireless LAN(Local Area Network) or Bluetooth (registered trademark). Further, itmay be possible to pass the image information to the image processingdevice 10 via a memory card, such as an SD card, or the like, withoutconnecting the image processing device 10 and the camera 40.

In such an image processing system 1, first, a user takes a photographof a photographing subject S by the camera 40. The image taken by thecamera 40 is an original image, and image information of the originalimage is sent to the image processing device 10. On the display device20, the original image, which is an image before color adjustment, isdisplayed. Next, when the user inputs instructions for performing coloradjustment to the image processing device 10 by use of the input device30, color adjustment is performed on the original image by the imageprocessing device 10. The result of the color adjustment is, forexample, is reflected in an image to be displayed on the display device20, and thereby, an image after color adjustment, which is differentfrom the image before color adjustment, is rendered again and displayedon the display device 20.

Moreover, as the image after color adjustment, other than the imageafter color adjustment by the image processing device 10, another imagephotographed by another camera having model properties different fromthose of the camera 40, that is, different photographing conditions, maybe adopted. In this case, the image taken by the camera 40 can beassumed to be the image before color adjustment, and another image takenby a camera with different photographing conditions can be assumed to bethe image after color adjustment.

Moreover, in the image processing device 10, based on the result of thecolor adjustment, a color conversion property (a color conversion model)is created. Here, “color conversion model” refers to a relationshipbetween image information before color adjustment and image informationafter color adjustment. Moreover, it can be said that “color conversionmodel” is a function representing the relationship between imageinformation before color adjustment and image information after coloradjustment. Moreover, though, to be described in detail later, uponconsidering a color conversion vector with image information beforecolor adjustment as a starting point and image information after coloradjustment as an ending point, it is also possible to say that “colorconversion model” is an aggregate of color conversion vectors (colorconversion vector group). For example, when image information is RGBdata composed of Red (R), Green (G) and Blue (B), assuming that imageinformation before color adjustment is represented as (R_(a), G_(a),B_(a)) and image information after color adjustment is represented as(R_(b), G_(b), B_(b)), a color conversion model represents arelationship between (R_(a), G_(a), B_(a)) and (R_(b), G_(b), B_(b)).

Further, in the image processing device 10, based on the colorconversion model, a conversion relationship, which performs colorconversion from image information of an original image before coloradjustment into image information after color adjustment, is created.Here, “conversion relationship” refers to conversion information forconverting image information before color adjustment into imageinformation after color adjustment. The conversion relationship can becreated as an LUT (Lookup Table). The LUT can be a multi-dimensionalLUT. Moreover, the LUT can also be a one-dimensional LUT. Further, theconversion relationship may be a multi-dimensional matrix, not an LUT.The conversion relationship may be retained as, other than themulti-dimensional LUT, the LUT and the matrix, teacher data(input-output data pair) for learning.

The conversion relationship is, when image information is RGB data,information for converting (R_(a), G_(a), B_(a)), which is imageinformation before color adjustment, into (R_(b), G_(b), B_(b)), whichis image information after color adjustment, namely, (R_(a), G_(a),B_(a))→(R_(b), G_(b), B_(b)). By use of the conversion relationship,color adjustment similar to previously performed color adjustment can bereproduced. In other words, when image information before coloradjustment is newly generated, it is possible to perform coloradjustment similar to previously performed color adjustment, to therebycreate image information after color adjustment by performing colorconversion by use of the conversion relationship.

When the conversion relationship is represented by a multi-dimensionalLUT, here, the relationship is represented by a three-dimensional LUT,to directly convert (R_(a), G_(a), B_(a)) into (R_(b), G_(b), B_(b)). Inother words, the conversion relationship performs (R_(a), G_(a),B_(a))→(R_(b), G_(b), B_(b)). Moreover, when the conversion relationshipis represented by a one-dimensional LUT, each of R, G, B is converted.In other words, the conversion relationship performs R_(a)→R_(b),G_(a)→G_(b), and B_(a)→B_(b). In the exemplary embodiments, conversionin the RGB color space is exemplified; however, conversion in othercolor spaces, such as CMYK, may be adopted. In this case, imageinformation is CMYK data composed of respective colors of C (cyan), M(magenta), Y (yellow) and K (black). Then, when the conversionrelationship is the multi-dimensional LUT, the LUT is a four-dimensionalLUT performing conversion of image information before color adjustment(C_(a), M_(a), Y_(a), K_(a)) into image information after coloradjustment (C_(b), M_(b), Y_(b), K_(b)), namely, (C_(a), M_(a), Y_(a),K_(a))→(C_(b), M_(b), Y_(b), K_(b)). Moreover, when the conversionrelationship is represented by a one-dimensional LUT, each of C, M, Y, Kis converted. In other words, the conversion relationship performsC_(a)→C_(b), M_(a)→M_(b), Y_(a)→Y_(b) and K_(a)→K_(b).

Note that the image processing system 1 in the exemplary embodiments isnot limited to the mode in FIG. 1. For example, as the image processingsystem 1, a tablet terminal can be exemplified. In this case, the tabletterminal includes a touch panel, which displays images and accepts inputof user's instructions, such as a touch. In other words, the touch panelfunctions as the display device 20 and the input device 30. Note that,as the camera 40, a built-in camera in the tablet terminal can be used.Moreover, similarly, as a device integrating the display device 20 andthe input device 30, a touch monitor can be used. This is the touchpanel used as the above-described display screen 20 a of the displaydevice 20. In this case, based on image information outputted by theimage processing device 10, an image is displayed on the touch monitor.A user inputs instructions for performing color adjustment by touchingthe touch monitor, or the like.

Here, when the image processing device 10 is to create the conversionrelationship, as described above, it is necessary to obtain imageinformation before color adjustment from an original image and to obtainimage information after color adjustment from an image after coloradjustment. In other words, it is necessary to obtain the imageinformation before color adjustment and the image information aftercolor adjustment corresponding thereto as an image information pair.

At this time, if an accuracy of plural pairs of image information beforecolor adjustment and image information after color adjustment is poor,it becomes difficult to create highly accurate color conversion model,and thereby an accuracy of the conversion relationship is reduced.Specifically, there are some cases in which, when the color adjustmentis to be performed, data of different directionality in color adjustmentis mixed, and those cases contribute to reduction in accuracy withrespect to the color conversion model derived from plural pairs of imageinformation. As factors causing the phenomenon include, for example, acase in which a user performing color adjustment has a problem withhis/her skill level and a case in which there are differences inenvironments to perform color adjustment. The differences inenvironments include, for example, differences in device properties ofthe display device 20 to be used and differences in illuminationenvironments. If data having different directionality in coloradjustment enters, directionality in color adjustment lacks integrity;therefore, it becomes difficult to create highly accurate colorconversion model. Then, when color adjustment is performed by aconversion relationship created as a result thereof, for example, coloradjustment that differs from a request is performed, or magnitude ofcolor adjustment is moderated, and thereby color adjustment with a smallamount of change is performed.

As described above, “accuracy” with respect to plural pairs of imageinformation before color adjustment and image information after coloradjustment is an index indicating whether or not the directionality incolor adjustment shows unity. Moreover, it can also be said that“accuracy” is an index indicating a degree of quality of plural pairs ofimage information before color adjustment and image information aftercolor adjustment in creating the color conversion model or theconversion relationship. Though details will be described later,“accuracy” can be quantified based on a color conversion vector withimage information before color conversion as a starting point and imageinformation after color conversion as an ending point, to be evaluated.Here, “accuracy” may be targeted at entirety of the plural pairs ofimage information pieces serving as a base for creating a colorconversion model, or targeted at part of the plural pairs of imageinformation pieces serving as a base for creating a color conversionmodel.

In this manner, a highly accurate color conversion model is needed.However, it is difficult for a user to determine the accuracy. Moreover,when a highly accurate color conversion model cannot be achieved, it isdifficult to recognize where a problem is.

Therefore, the exemplary embodiments focus the existence of relationshipbetween the accuracy of the color conversion model and the accuracy ofplural pairs of image information before color adjustment and imageinformation after color adjustment, and cause the above-describedproblem to hardly occur by configuring the image processing device 10 asfollows. In other words, by evaluating the accuracy with respect to theplural pairs of image information before color adjustment and imageinformation after color adjustment, the accuracy of the color conversionmodel created based on the plural pairs of image information pieces isquantitatively evaluated.

Note that, hereinafter, items first inputted by a user as an imagebefore color adjustment and an image after color adjustment are referredto as “first image” in some cases. It can also be said that “firstimage” is a basic set of images as pairs of an image before coloradjustment and an image after color adjustment. Moreover, as an imagebefore color adjustment and an image after color adjustment, itemsinputted by a user for confirming the accuracy with respect to pluralpairs of image information before color adjustment and image informationafter color adjustment are referred to as “second image” in some cases.It can also be said that “second image” is a set of images forevaluating the accuracy by a user as pairs of an image before coloradjustment and an image after color adjustment. Further, items inputtedby a user in addition to the first image as an image before coloradjustment and an image after color adjustment are referred to as “thirdimage” in some cases. It can also be said that “third image” is a set ofimages added to the first image as pairs of an image before coloradjustment and an image after color adjustment.

<Description of Image Processing Device 10 and Display Device 20>

Next, the image processing device 10 and the output device 20 will bedescribed.

FIG. 2 is a block diagram showing a functional configuration example ofthe image processing device 10 and the display device 20 in theexemplary embodiments. Note that, in FIG. 2, of the various functionsincluded in the image processing device 10 and the display device 20,those related to the exemplary embodiment are selected and shown.

As shown in the figure, the image processing device 10 of the exemplaryembodiments includes: a first image obtaining unit 11 that obtains firstimage information; an image information obtaining unit 12 that obtainsan image information pair from the first image; a first accuracyevaluation display 13 that displays an image for evaluating accuracy ofplural pairs of image information pieces; a second image obtaining unit14 that obtains image information of a second image for evaluation; asecond accuracy evaluation display 15 that displays an image forevaluating accuracy of plural pairs of image information pieces based onthe image information of the second image; a user's evaluationacceptance unit 16 that accepts user's evaluation; an additionalinformation deriving unit 17 that requests information of an additionalimage when addition of an image is required; a third image obtainingunit 18 that obtains image information of a third image, which is anadditional image; and a color conversion coefficient calculation unit 19that calculates a color conversion coefficient as a conversionrelationship.

Moreover, the display device 20 includes: a display informationobtaining unit 21 that obtains display information for displaying animage from the image processing device 10; and a display 22 thatdisplays an image based on the display information. The display 22corresponds to the display screen 20 a described above.

The first image obtaining unit 11 obtains image information of each ofthe image before color adjustment and the image after color adjustment,which are the first image.

These image information pieces have a data format for executing displayon the display device 20, which is, for example, the above-described RGBdata. Note that the first image obtaining unit 11 may obtain imageinformation in other data formats and perform color conversion to createthe RGB data or the like.

Consequently, the first image obtaining unit 11 functions as anacceptance unit that accepts image information pairs, each of which iscomposed of image information before color conversion and imageinformation after color conversion.

FIGS. 3A to 3F are diagrams showing examples of the first image obtainedby the first image obtaining unit.

Here, a case is shown in which three pairs of image information pieceswhen photographing clothing that is a commercial product or a personwearing clothing that is a commercial product are prepared. Of these,each of FIGS. 3A and 3B is a first image when a down jacket Dj isphotographed. Here, FIG. 3A shows an image before color adjustment andFIG. 3B shows an image after color adjustment. Similarly, each of FIGS.3C and 3D is a first image when a person wearing a shirt Sh isphotographed. Here, FIG. 3C shows an image before color adjustment andFIG. 3D shows an image after color adjustment. Further, each of FIGS. 3Eand 3F is a first image when a person wearing a dress Dr isphotographed. Here, FIG. 3E shows an image before color adjustment andFIG. 3F shows an image after color adjustment.

As described above, in the exemplary embodiments, by preparing pluralpairs of the image before color adjustment and the image after coloradjustment, image information pairs including more colors are obtained.

The image information obtaining unit 12 includes, as shown in FIG. 2: aregion determination unit 121 that determines regions of the imagebefore color adjustment and the image after color adjustment, from whichimage information is to be extracted; an image information extractionunit 122 that extracts image information pairs; an image informationmemory 123 that stores the extracted image information pairs; and anaccuracy derivation unit 124 that calculates accuracy of the imageinformation pairs.

The region determination unit 121 determines, of any one of the imagebefore color adjustment and the image after color adjustment, a regionfrom which image information is to be extracted.

In other words, the region determination unit 121 determines, forexample, from the images shown in FIGS. 3A to 3F, from which positionthe image information is to be obtained. In this case, color adjustmentis performed on the locations of clothing, which is a commercialproduct. In other words, as to colors of a commercial product, it isrequired to perform color reproduction more precisely, to thereby tomatch the color of the commercial product displayed as an image with thecolor of the actual commercial product. Therefore, a color of acommercial product is likely to be a subject of color adjustment.

Specifically, for example, the region determination unit 121 determinesa location other than a background to be a region from which imageinformation is to be extracted. Therefore, the region determination unit121 is required to discriminate between the background and a locationother than the background. Here, image information of the background isalmost the same as image information at the left end of the image.Therefore, it is possible to assume a location, in which imageinformation significantly changes from the image information of the leftend of the image, to be the location other than the background. At thistime, for sampling image information to be compared with the imageinformation at the left end of the image, for example, pixel positionsare determined in the image at predetermined intervals, and imageinformation of each of the pixels is compared with the image informationof the pixel at the left end of the image. Moreover, it may be possiblethat a mask of a predetermined size is applied to image information andan average value of image information within the mask is compared withthe image information of the pixel at the left end of the image.

Further, as another method, a frequency analysis is conducted based onimage information to obtain pixel positions where a high frequency isgenerated. Since the pixel positions become an outline of portions otherthan the background, inside the outline is assumed to be the portionsother than the background. Further, as still another method, a range ofa predetermined size from the center of an image is prescribed, andinside the range is assumed to be portions other than the background.

The region determination unit 121 performs above-described processing onany one of the image before color adjustment and the image after coloradjustment, to determine a region from which image information is to beextracted.

The image information extraction unit 122 extracts image informationpieces from within a region of one of the image before color adjustmentand the image after color adjustment, which is designated by the regiondetermination unit 121, and from within a region of the other imagecorresponding thereto. This can also be said that, as the imageinformation pair at the positions corresponding to each other in theimages, image information is extracted from the image before coloradjustment and image information is extracted from the image after coloradjustment.

In other words, from the image before color adjustment and the imageafter color adjustment, at the positions in these images, imageinformation before color adjustment and image information after coloradjustment are extracted.

FIGS. 4A and 4B are diagrams showing an example of pairs of imageinformation before color adjustment and image information after coloradjustment.

Here, FIG. 4A shows an example of an image information before coloradjustment and image information before color adjustment extracted fromthe image. Here, the image before color adjustment is an image of ablouse, and image information pieces extracted at locations indicated byBr1 to Br5 inside the image are indicated as RGBa1 to RGBa5. In thiscase, the blouse is in a solid blue color, and any of RGBa1 to RGBa5 isRGB data indicating the blue color.

Moreover, FIG. 4B shows an example of an image information after coloradjustment and image information after color adjustment extracted fromthe image. Here, image information pieces extracted at locationsindicated by Br1 to Br5, which are similar to those in FIG. 4A, areindicated as RGBb1 to RGBb5.

In the image information obtaining unit 12, by the method as describedabove, the image information before color conversion and the imageinformation after color conversion corresponding thereto are obtained asimage information pairs. The obtained image information pairs are storedin the image information memory 123.

The accuracy derivation unit 124 calculates the accuracy with respect toplural image information pairs extracted in the image informationextraction unit 122.

The accuracy derivation unit 124 calculates the accuracy based on acolor conversion vector with image information before color conversionas a starting point and image information after color conversion as anending point.

FIGS. 5A and 5B are diagrams showing methods of obtaining the accuracyfrom color conversion vectors.

Of these, FIG. 5A shows a case in which the accuracy is derived from anangle between plural color conversion vectors.

In the shown example, as the color conversion vectors, two vectors, acolor conversion vector Vc1 and a color conversion vector Vc2, areshown. The starting points of these color conversion vectors Vc1 and Vc2are, as described above, the image information before color conversion(color values), and the ending points mean the image information aftercolor conversion (color values). In other words, the color conversionvectors Vc1 and Vc2 indicate a moving direction and a moving amount ofimage information by the color conversion. Then, let us consider theangle θ between the color conversion vector Vc1 and the color conversionvector Vc2. When, in a specific color region, there is uniteddirectionality and integrity in color adjustment, the plural colorconversion vectors are in the same direction; therefore, the angle θ ismore likely to approach zero. In contrast thereto, when thedirectionality is not united and the integrity is poor in coloradjustment, directions of the plural color conversion vectors are lessprone to be the same; therefore, the angle θ is more likely to beincreased. In other words, when the accuracy with respect to the pluralimage information pairs obtained in the image information obtaining unit12 is high, the angle θ between the plural color conversion vectors ismore likely to be reduced, whereas, when the accuracy is not high, theangle θ between the plural color conversion vectors is more likely to beincreased. Accordingly, the above-described accuracy can be derived fromthe angle θ between the plural color conversion vectors.

Moreover, FIG. 5B shows a case in which the accuracy is derived from acolor difference between ending points of plural color conversionvectors.

In the shown example, as the color conversion vector, a color conversionvector Vc3 is shown. Moreover, for the image information of the imagebefore color conversion and the image information after color conversionused as the color conversion vector Vc4, learning data, which will bedescribed later, may be used or non-learning data may be used. Then, letus consider a difference (color difference) L between the imageinformation at the ending point of the color conversion vector Vc3 andthe image information after color conversion applied to the imageinformation of the image before the color conversion and serving as thestarting point based on the color conversion property. Here, let usconsider the difference (color difference) L between the imageinformation pieces at the ending points of the color conversion vectorVc3 and the color conversion vector Vc4. Note that the color differenceL can also be considered to be the Euclidean distance in the color spacerepresenting the image information (for example, RGB color space). When,in a specific color region, there is aligned directionality andintegrity in color adjustment, these color conversion vectors Vc3 andVc4 are in the same direction and in the same magnitude; therefore, thepositions of the ending points are unlikely to be varied. As a result,the color difference L is more likely to be reduced. In contrastthereto, when the directionality is not aligned and the integrity ispoor in color adjustment, directions and magnitudes of these colorconversion vectors Vc3 Vc4 are less prone to be the same, and therefore,the positions of the ending points are likely to be varied. As a result,the color difference L is more likely to be increased. In other words,when the accuracy with respect to the plural image information pairsobtained in the image information obtaining unit 12 is high, the colordifference between the ending points of these color conversion vectorsVc3 and Vc4 is more likely to be reduced, whereas, when the accuracy isnot high, the color difference between the ending points of these colorconversion vectors Vc3 and Vc4 is more likely to be increased.Accordingly, the above-described accuracy can be derived from the colordifference between the ending points of these color conversion vectorsVc3 and Vc4.

Note that the image information obtaining unit 12 can exclude a colorconversion vector having an angle or a color difference significantlydifferent from those of other color conversion vectors. That is, it ispossible to exclude image information pairs generating such colorconversion vectors. In other words, the color adjustment causing suchimage information pairs has directionality of color adjustment that isapparently different; accordingly, it is inappropriate to deriveaccuracy or to create a highly accurate color conversion model.Therefore, the image information obtaining unit 12 can also exclude suchimage information pairs from the image information pairs. To determinewhether or not a color conversion vector has an angle or a colordifference significantly different from those of other color conversionvectors, existing statistical methods can be used.

Moreover, the image information obtaining unit 12 can be grasped as anaccuracy output unit that outputs accuracy of a color conversionproperty from plural image information pairs accepted by the first imageobtaining unit 11 or the third image obtaining unit 18.

The first accuracy evaluation display 13 creates accuracy information todisplay accuracy calculated in this manner. Control is performed tooutput and display the display information onto the display device 20.

FIG. 6 is a diagram showing an example in which display informationcreated by the first accuracy evaluation display 13 is displayed on thedisplay 22 of the display device 20.

The shown example displays an entire color region 221 from which theaccuracy is calculated in a left-side area RL. Here, the entire colorregion 221 is displayed by two annular rings 221 a and 221 b.

In the example shown in FIG. 6, the entire color region 221 is dividedinto eight color regions, the region 1 to the region 8. Note that “colorregion” refers to each of the regions when a color space is divided by apredetermined method. Here, an entire color space to be used is dividedby a predetermined rule, and each of the divided regions is assumed tobe a color region. More specifically, predetermined boundary values areprovided to hue, saturation and brightness, and thereby the dividedregions can be set as the respective regions. In FIG. 6, as an exampleof this, regions divided based on hue and saturation are set as therespective regions.

Then, in the exemplary embodiments, the first accuracy evaluationdisplay 13 calculates the accuracy for each of the color regions.

In the example shown in FIG. 6, numerals 1 to 4 are displayed on theinside annular ring 221 a, which indicate that the accuracy iscalculated in the region 1 to the region 4 of the entire color region221. Moreover, numerals 5 to 8 are displayed on the outside annular ring221 b, which indicate that the accuracy is calculated in the region 5 tothe region 8 of the entire color region 221. Moreover, each of theregion 1 to the region 8 is painted by actual color in the region.Consequently, the two annular rings 221 a and 221 b represents a colorregion by combining the accuracy displayed in each region.

Moreover, in the shown example, in a right-side area RR, informationincluding the accuracy in the regions 1 and 7 of the entire color region221 is displayed.

In the right-side area RR, in an area 222, information about a colorregion with low accuracy is displayed. Here, it is assumed that theaccuracy is low in the region 1.

In the area 222, the color in the region 1 is displayed as a colorsample 222 a, and “region 1” is displayed as region information 222 bindicating information related to the region 1. Further, in the area222, the accuracy 222 c of plural pairs of image information isdisplayed, and a sufficiency level of image information pairs isdisplayed as a number of data items 222 d. This “sufficiency level” is aratio of image information pairs actually inputted by the first imageobtaining unit 11 to the required image information pairs. There aresome cases in which “sufficiency level” is determined across the boardfor each of the color regions and some other cases in which “sufficiencylevel” is individually determined for each of the color regions. In theexemplary embodiments, the sufficiency level is determined by a methodthat individually determines the level for each of the color regions.

Of these, the accuracy 222 c is displayed as “Level”, including fivelevels of A, B, C, D and E. In this case, the highest accuracy is A andthe lowest accuracy is E. Moreover, here, the accuracy 222 c isdisplayed for two cases, “Before” and “After”. The case “Before” is theaccuracy of plural pairs of image information related to the firstimage, and, here, it is indicated that the Level is C. Moreover, thoughdetails will be described later, “After” is the accuracy of plural pairsof image information after the third image, which is an additionalimage, is added, and, here it is indicated that the Level is B. In otherwords, here, it can also be said that the accuracy of the colorconversion property before and after new image information pairs (thethird image) are added is displayed on the display device 20.

Moreover, the number of data items 222 d is displayed by a number ofcolored boxes in five levels. In other words, it is indicated that, theless the number of colored boxes is, the lower the sufficiency level ofthe number of image information pairs is, whereas, the more the numberof colored boxes is, the higher the sufficiency level of the number ofimage information pairs is. Moreover, here, the number of data items 222d is also displayed for two cases, “Before” and “After”. The case“Before” is the sufficiency level of the number of image informationpairs related to the first image, and, here, it is indicated that thenumber of data items 222 d is on the third level in the five levels.Moreover, though details will be described later, “After” is thesufficiency level of the number of image information pairs after thethird image, which is an additional image, is added, and, here it isindicated that the number of data items 222 d is on the third level inthe five levels.

Moreover, in the area 223, for comparison with the low-accuracy region1, information about a high-accuracy color region is displayed. Here, itis assumed that the accuracy is high in the region 7.

In the area 223, information similar to that in the area 222 isdisplayed for the region 7. In other words, in the area 223, the colorin the region 7 is displayed as a color sample 223 a, and regioninformation 223 b indicating information related to the region 7 isdisplayed. Further, in the area 223, the accuracy 223 c of plural pairsof image information is displayed, and the sufficiency level of imageinformation pairs is displayed as a number of data items 223 d.

As described above, in the display 22, the accuracy is displayed foreach color region. Here, in the areas 222 and 223, the regions 1 and 7are selected from the color regions, the region 1 to the region 8, tothereby display the accuracies 222 c and 223 c. Moreover, in the display22, the accuracies are respectively displayed before and after the imageinformation of the third image, which obtains additional imageinformation pairs, is obtained. Here, in the areas 222 and 223, this isperformed by displaying “Before” and “After” in the accuracies 222 c and223 c, respectively.

Moreover, in the display 22, the sufficiency level of the imageinformation pairs required to create the color conversion model isdisplayed. Here, in the areas 222 and 223, as the sufficiency level ofrequired the image information pairs, the number of data items 222 d and223 d are displayed. Moreover, in the display 22, the sufficiency levelsof the image information pairs are respectively displayed before andafter the image information of the third image, which obtains additionalimage information pairs, is obtained. Here, in the areas 222 and 223,this is performed by displaying “Before” and “After” in the number ofdata items 222 d and 223 d, respectively.

Then, in the area 224, as to each of the region 1 and the region 7, theobtained number of image information pairs is displayed as the number ofimages of the first image 224 a. Here, it is indicated that the numberof images of the first image 224 a in the region 1 is 20 and the numberof images of the first image 224 a in the region 7 is 100.

Further, in the area 224, as to each of the region 1 and the region 7,the degree of variation of the color conversion vectors is displayed asimage uniformity 224 b. Here, the image uniformity 224 b is displayed,for example, in five levels of A, B, C, D, E from good to bad. It isindicated that the image uniformity 224 b in the region 1 is B and theimage uniformity 224 b in the region 7 is A.

Further, a button 225 is selected by a user when the above-describedthird image is inputted.

Returning to FIG. 2, the second image obtaining unit 14 obtains imageinformation of a second image by which a user confirms accuracy. Thesecond image is an image for evaluation by which the user confirmsaccuracy. Moreover, here, similar to the first image, the second imageincludes pairs each being composed of an image before color adjustmentand an image after color adjustment. Consequently, similar to the firstimage obtaining unit 11, the second image obtaining unit 14 functions asan acceptance unit that accepts image information pairs each beingcomposed of image information before color conversion and imageinformation after color conversion.

The second accuracy evaluation display 15 performs, regarding the secondimage, control to further create display information for displaying animage before color conversion and an image after color conversion basedon a color conversion model. Then, the display 22 displays theinformation. In other words, here, a tentative color conversion model iscreated based on the image information pairs obtained in the first imageobtaining unit 11, and a result of color adjustment that can beperformed by use of the color conversion model is assumed to be an imageafter color conversion, and displayed on the display 22. The userconfirms the above-described accuracy by observing the image.

FIG. 7 is a diagram showing a first example in which display informationcreated by the second accuracy evaluation display 15 is displayed on thedisplay 22 of the display device 20.

In FIG. 7, “color region” is determined to be a specific color. Thespecific color is not particularly limited, and is determined inaccordance with a user's purpose. In this case, the determined colorregion may not cover the entire color space to be used, but may coveronly a part thereof. For example, when color adjustment is performedonly for a beige color, the solid beige color may be determined as thecolor region. Moreover, for example, when color adjustment is performedonly for metal parts, a solid metallic color may be determined as thecolor region. Of course, plural color regions may be determined. In FIG.7, as an example, it is assumed that six color regions of red, white,gray, beige, blue and yellow are determined.

In the exemplary embodiments, the second accuracy evaluation display 15also calculates the accuracy for each of the color regions.

The shown example displays, from among the entire color region, a colorregion, for which an image for confirming accuracy is displayed, in theleft-side area RL. Here, in the area 226, the color in the color regionof gray is displayed as a color sample 226 a, and “region: gray” isdisplayed as region information 226 b indicating information related tothe color region of gray. Further, in the area 226, the accuracy 226 cof plural pairs of image information is displayed, and the sufficiencylevel of image information pairs is displayed as the number of dataitems 226 d.

Moreover, similar to the case shown in FIG. 6, the accuracy 226 c isdisplayed as “Level”, and is displayed for two cases, “Before” and“After”. Here, it is indicated that “Level” is C in both cases.Moreover, similar to the case shown in FIG. 6, the number of data items226 d is displayed by a number of colored boxes in five levels, and isdisplayed for two cases, “Before” and “After”. It is indicated that, asto “Before”, the number of data items 226 d is on the third level in thefive levels, and as to “After”, the fifth level in the five levels.

Moreover, here, in the area 227, the accuracy of the high-accuracyregion is displayed for comparison. Here, the color in the region of redis displayed as a color sample 227 a, and “region: red” is displayed asregion information 227 b indicating information related to the region ofred. Then, as the accuracy 227 c, it is indicated that “Level” in theregion of red is A. Further, it is indicated that the number of dataitems 227 d is on the fifth level in the five levels.

Note that, in the left-side area RL, the button 225 having the functionsimilar to the case shown in FIG. 6 is displayed.

Moreover, in the shown example, in an area 228 of the right-side areaRR, an image for confirming the accuracy by a user in the color regionof gray is displayed. Here, of the second image obtained in the secondimage obtaining unit 14, the image before color adjustment is displayedas an image “before correction” 228 a. Moreover, of the second imageobtained in the second image obtaining unit 14, the image after coloradjustment is displayed as a “target” image 228 c. Then, between theimage 228 a and the image 228 c, a result of color adjustment that canbe performed by use of a tentative color conversion model is displayedas an image of “automatic correction” 228 b.

Moreover, in the area 229 of the right-side area RR, an image forconfirming the accuracy by a user in the color region of red isdisplayed. The image displayed in the area 229 is displayed by a methodsimilar to that in the area 228. In other words, of the second imageobtained in the second image obtaining unit 14, the image before coloradjustment is displayed as an image “before correction” 229 a. Moreover,of the second image obtained in the second image obtaining unit 14, theimage after color adjustment is displayed as a “target” image 229 c.Then, between the image 229 a and the image 229 c, a result of coloradjustment that can be performed by use of a tentative color conversionmodel is displayed as an image of “automatic correction” 229 b.

The user compares the image 228 b and the image 228 c in the area 228,and thereby, it is possible to determine the accuracy of theabove-described tentative color conversion model. Similarly, the usercompares the image 229 b and the image 229 c in the area 229, andthereby, it is possible to do the same. However, the area 228 displays acomparison in the case of the low accuracy, whereas, the area 229displays a comparison in the case of the high accuracy. Therefore, bycomparing the area 228 with the area 229, the user can also compare thelow-accuracy case and the high-accuracy case.

Note that, since the accuracy of the tentative color conversion modeland the accuracy of plural pairs of image information are related toeach other, it can be considered that these indicate the accuracy of theplural pairs of image information.

Note that, here, the second image is pairs of the image before coloradjustment and the image after color adjustment; however, it may bepossible to omit the image after color adjustment, and to leave only theimage before color adjustment. In this case, “target” images 228 c and229 c, which are the images after color adjustment, are not displayed.In this case, the images 228 a and 228 b in the area 228 and the images229 a and 229 b in the area 229 are compared in each area, to therebydetermine the accuracy of the tentative color conversion model.

FIG. 8A is a diagram showing a second example in which displayinformation created by the second accuracy evaluation display 15 isdisplayed on the display 22 of the display device 20.

In the shown example, in the area 226 of the left-side area RL, an imagesimilar to that in the area 226 in FIG. 7 is displayed. In other words,as to the color region of gray, the color sample 226 a, the regioninformation 226 b, the accuracy 226 c and the number of data items 226 dare displayed.

Moreover, in the left-side area RL, the button 225 having the functionsimilar to the button 225 shown in FIG. 7 is displayed.

Moreover, in the area 230 of the right-side area RR, an image forconfirming the accuracy by a user in the color region of gray isdisplayed. Then, similar to the case of FIG. 7, the image before coloradjustment is displayed as an image “before correction” 230 a. On theother hand, here, the result of color adjustment before the third imageis added is displayed as an image 230 b of “Learning 1”, and the resultof color adjustment after the third image is added is displayed as animage 230 c of “Learning 2”. In other words, it can be considered thatthe image 230 b and the image 230 c correspond to “Before” and “After”of the accuracy 226 c.

Note that, here, images of the area 227 and the area 229 in FIG. 7 arenot displayed; however, the same images as those may also be displayedin the case of FIG. 8A. Moreover, portions displaying the images in theareas 228 and 229 in FIG. 7 or the areas 230 and 232 in FIG. 8A maydisplay diagrams (or fills) of RGB values, not the images.

FIG. 8B is a diagram showing a third example in which displayinformation created by the second accuracy evaluation display 15 isdisplayed on the display 22 of the display device 20.

The shown example indicates an example in which an image, by which auser confirms accuracy as to an entire color region, is displayed. Here,in the area 231 of the left-side area RL, the accuracy 231 c and thenumber of data items 231 d are displayed.

The accuracy 231 c is also displayed here as “Level”, including fivelevels of A, B, C, D and E. Moreover, here, two sets of the third image,which is the additional image, are prepared and assumed to be “Learning1” and “Learning 2”, for each of which the accuracy 231 c is displayed.In this case, as the accuracy of “Learning 1”, it is indicated that“Level” is A. Moreover, as the accuracy of “Learning 2”, it is indicatedthat “level” is C.

The number of data items 231 d is, similar to the case shown in FIG. 7,displayed by the number of colored boxes in five levels, and isdisplayed for two cases, “Learning 1” and “Learning 2”. Here, it isindicated that, as to “Learning 1”, the number of data items 231 d is onthe fourth level in the five levels, and as to “Learning 2”, the fifthlevel in the five levels.

Moreover, here, in the area 232 of the right-side area RR, an image forconfirming the accuracy in the entire color region by a user isdisplayed. Then, similar to the case of FIG. 8A, the image before coloradjustment is displayed as an image “before correction” 232 a. Moreover,here, as the images for confirming the accuracy of the prepared two setsof the third image, an image 232 b of “Learning 1” and an image 232 c of“Learning 2” are displayed. In other words, it can be considered thatthe image 232 b and the image 232 c correspond to “Learning 1” and“Learning 2” of the accuracy 231 c. Moreover, here, an image 232 d of“Target”, which is an image after color adjustment, is displayed;therefore, it is possible to compare and determine which one of theimage 232 b of “Learning 1” and the image 232 c of “Learning 2” iscloser to the image 232 d of “Target”. Further, as a result of thecomparison, the user can select any one of “Learning 1” and “Learning 2”as the learning to be used.

A button 233 is used when the user selects any one of “Learning 1” and“Learning 2”.

FIG. 9 is a diagram showing a fourth example in which displayinformation created by the second accuracy evaluation display 15 isdisplayed on the display 22 of the display device 20.

Here, a case where accuracies in red, white, gray, beige, blue andyellow, which are set as the color regions, are displayed in a list isshown.

Here, in the left-side area RL, the colors of the respective colorregions are displayed as color samples 234 a, and names of therespective color regions are displayed as region information items 234b.

Further, in the right-side area RR, the accuracies 234 c are displayed.The accuracy 234 c is displayed as “Level”, and display is performed,not by the number of boxes, but by numerals of 1, 2, 3, 4, 5 in fivelevels. In this case, the highest accuracy is 5 and the lowest accuracyis 1. Then, “Level” before the third image is added is displayed as“Learning (initial)”, and “Level” after the third image is added isdisplayed as “Re-learning”.

The first accuracy evaluation display 13 and the second accuracyevaluation display 15 can be grasped as a display information creationunit that, when the color conversion model that converts the imageinformation of an image before color conversion into the imageinformation of the image after color conversion is created, createsdisplay information that displays accuracy of plural image informationpairs. Moreover, the first accuracy evaluation display 13 and the secondaccuracy evaluation display 15 can be grasped as a display control unitthat controls to display the accuracy outputted by the image informationobtaining unit 12 for each of the color regions on the display device20. Further, the second accuracy evaluation display 15 can be grasped asa display control unit that, when the second image obtaining unit 14accepts new image information pairs (the second image), controls todisplay, on the display device 20, at least image information created bycolor conversion of the image information of the new image informationpairs (the second image) before color conversion based on the colorconversion property (color conversion model) created by the colorconversion coefficient calculation unit 19 from the image informationpairs having already been accepted by the first image obtaining unit 11and image information of the image information pairs (the second image)after color conversion.

When an image is displayed on the display 22 based on the displayinformation of the second image, the user's evaluation acceptance unit16 accepts user's evaluation about color conversion by the colorconversion model.

In other words, when the user observes the images in the area 228 andthe area 229 in FIG. 7, evaluation whether or not the accuracy of theabove-described tentative color conversion model can be allowed isaccepted. At this time, the display 22 displays the user's evaluationacceptance screen that accepts user's evaluation of color conversion bythe color conversion model, as a result of displaying the second image.

FIG. 10 is a diagram showing an example in which the user's evaluationacceptance screen is displayed on the display 22.

The shown example indicates the user's evaluation acceptance screendisplayed as a window W1 on the display 22. Here, a case, in whichevaluation of the color region of gray in FIG. 7 is inputted, is shown.In other words, a case, in which a user compares the images 228 a to 228c with one another in the area 228 in FIG. 7, and evaluation thereby isinputted, is shown. Then, on the window W1, the message Me1 isdisplayed, “Please input evaluation of gray.” and the user can inputevaluation in five levels of 1, 2, 3, 4, 5 from the window W1. In thiscase, the lowest evaluation is 1 and the highest evaluation is 5. Then,by selecting one of radio buttons Rb adjacent to the respectivenumerical values 1 to 5, the user can input evaluation of any one of 1to 5. Here, an example is shown in which the user selects a radio buttonRb corresponding to 3 as the evaluation.

Note that, in FIG. 10, the five levels of 1 to 5 are set as the user'sevaluation; however, the number of levels is not particularly limited.For example, two levels of “good” or “bad” can be set. Moreover, here, acase of inputting the user's evaluation of the color region of gray isshown; however, the user's evaluation may be inputted for other colorregions. Further, it may be possible that the user inputs plural secondimages, and provides user's evaluation for each of the plural secondimages. Further, it may be possible to discriminate among the colorregions included in the second image, determine a representative imagein each of the color regions from the second image and display thereofin the area 228 or the area 229 in FIG. 7, and input user's evaluationbased thereon.

Further, evaluation may be performed by selecting one by the user fromthe images for confirming the accuracy of the prepared plural sets ofthe third image as shown in FIG. 8. Consequently, it can be said thatthe screen shown in FIG. 8 is the user's evaluation acceptance screen.

When the image information pairs are insufficient, the additionalinformation deriving unit 17 obtains color regions required for imageinformation pairs to be added. In other words, the additionalinformation deriving unit 17 obtains the color regions required for thethird image including pairs each being composed of the image beforecolor adjustment and the image after color adjustment. At this time, theadditional information deriving unit 17 creates display information thatdisplays an additional information screen for displaying the colorregion required for image information pairs to be added. Then, thedisplay 22 displays the additional information screen.

Moreover, the additional information deriving unit 17 can further obtainthe number of pairs necessary as the third image. In this case, thedisplay 22 further displays the number of additional pairs necessary asthe third image.

FIG. 11 is a diagram showing an example in which the additionalinformation screen is displayed on the display 22.

The shown example indicates the additional information screen displayedas a window W2 on the display 22. Then, on the window W2, the messageMe2 is displayed, “Images for learning are too few in number. Please setmore images.”

Moreover, on the window W2, the sufficiency level of the imageinformation pairs is displayed by a number of colored boxes Bx in fourlevels for each color region. In other words, it is indicated that, theless the number of colored boxes Bx is, the lower the sufficiency levelof the number of image information pairs is, whereas, the more thenumber of colored boxes Bx is, the higher the sufficiency level of thenumber of image information pairs is. Further, on the window W2, thenumber of pairs to be added Tm is displayed for each color region.

It is possible to set the number of images necessary to each colorregion in advance, and to assume the number of pairs to be added Tm as adifference from the number of first images actually inputted. Moreover,the number of pairs to be added Tm may be increased or decreased by theabove-described accuracy or user's evaluation, not simply by thedifference in number. In other words, when the accuracy is high, thenumber of pairs to be added Tm is decreased. In contrast thereto, whenthe accuracy is low, the number of pairs to be added Tm is increased.Moreover, when the user's evaluation is high, the number of pairs to beadded Tm is decreased. In contrast thereto, when the user's evaluationis low, the number of pairs to be added Tm is increased.

Here, an example, in which the boxes Bx and the number of pairs to beadded are displayed for the six color regions of red, white, gray,beige, blue and yellow, is shown. Here, it is indicated that, as to thecolor regions of beige and yellow, the image information pairs are notinsufficient, and thereby the number of pairs to be added is zero.Moreover, in each of the color regions of the other colors, the imageinformation pairs are insufficient and the required number of images isdisplayed. For example, as to the color region of gray, it is displayedthat the number of pairs to be added is 20. The user can grasp whetheror not the third image, which is an additional image, is necessary bylooking at the additional information screen. Further, the user cangrasp in which color region insufficiency occurs, and, for example, theuser can select an image including much color of the color region as thethird image.

The third image obtaining unit 18 obtains image information of the thirdimage. Consequently, similar to the first image obtaining unit 11 andthe second image obtaining unit 14, the third image obtaining unit 18functions as an acceptance unit that accepts image information pairseach being composed of image information before color conversion andimage information after color conversion.

This makes it possible to, for example, when the user's evaluationaccepted by the user's evaluation acceptance unit 16 is not higher thana predetermined criterion, obtain the image information pairs. At thistime, the display 22 displays a third image obtaining screen forobtaining image information of the third image.

Examples of the third image obtaining screen include the screens shownin FIGS. 6 and 7 displaying the button 225 selected by the user wheninputting the third image. Note that, other than this, a window forobtaining the third image may be displayed.

On the image information of the third image, processing similar to theprocessing on the image information of the first image is performed. Inother words, in the image information obtaining unit 12, the regiondetermination unit 121 determines a region from which the imageinformation is extracted, and the image information extraction unit 122extract the image information pairs. Then, the image information memory123 stores the extracted image information pairs, and the accuracyderivation unit 124 calculates the accuracy of plural image informationpairs. Note that the accuracy means accuracy of the plural pairs ofimage information after the third image is added to the imageinformation of the first image. The calculated accuracy is displayed atthe portion of “After” described in FIG. 6 or FIG. 7.

The color conversion coefficient calculation unit 19 creates the colorconversion model. Consequently, the color conversion coefficientcalculation unit 19 functions as a color conversion property creationunit that creates the color conversion property (color conversion model)from the plural image information pairs accepted by the first imageobtaining unit 11. Further, the color conversion coefficient calculationunit 19 creates the conversion relationship, such as thethree-dimensional LUT, based on the color conversion model.

The color conversion coefficient calculation unit 19 creates the colorconversion model based on the pairs of image information before coloradjustment and image information after color adjustment obtained fromthe first image or the third image. In other words, the color conversionmodel representing a relationship between the image information beforecolor adjustment and the image information after color adjustment iscreated.

FIG. 12 is a diagram showing an example of the color conversion model.

Here, the horizontal axis indicates the image information before coloradjustment and the vertical axis indicates the image information aftercolor adjustment. The image information before color adjustment and theimage information after color adjustment are the RGB data; in FIG. 12,the image information before color adjustment is shown as RGBa, and theimage information after color adjustment is shown as RGBb.

Black dots Pr indicate plotted image information before color adjustmentand image information after color adjustment; here, it is indicated thatthere are 12 pairs of image information before color adjustment andimage information after color adjustment.

Moreover, the solid line Js represents a color conversion modelindicating a relationship between the image information before coloradjustment and the image information after color adjustment, the colorconversion model being created by the color conversion coefficientcalculation unit 19. As mentioned above, it can be said that the colorconversion model is a function representing the relationship between theimage information before color adjustment and the image informationafter color adjustment, and supposing the function is f, the functioncan be expressed as RGBb=f(RGBa). The color conversion model can becreated by a publicly known method. However, it is preferred that amethod capable of highly fitting into non-linear characteristics, suchas a weighting regression model or a neural network, is used. However,not being limited to the non-linear characteristics, linearcharacteristics using Matrix model may be adopted.

Modified Example

Next, a modified example in the exemplary embodiments will be described.

FIG. 13 is a block diagram showing a modified example of a functionalconfiguration of the image processing device 10 and the display device20 in the exemplary embodiments. Note that, in FIG. 13, of the variousfunctions included in the image processing device 10 and the displaydevice 20, those related to the exemplary embodiment are selected andshown.

In the image processing device 10 and the display device 20 in theexemplary embodiments shown in FIG. 13, in addition to the componentsshown in FIG. 2, a classification unit 125 is provided to the imageinformation obtaining unit 12. Then, the second image obtaining unit 14obtains image information of the second image from the classificationunit 125. Moreover, others are similar to those shown in FIG. 2. In thiscase, functions of portions other than the classification unit 125 aresimilar to those in the case of FIG. 2. Consequently, hereinafter, theclassification unit 125 will be mainly described.

The classification unit 125 classifies the image information pairsextracted in the image information extraction unit 122 into the learningdata and the non-learning data. In this case, “learning data” is imageinformation pairs used for creating the color conversion model.Moreover, “non-learning data” is image information pairs not used forcreating the color conversion model. Then, the second accuracyevaluation display 15 assumes the non-learning data as the imageinformation of the second image by which a user confirms accuracy. Inother words, the image processing device 10 shown in FIG. 13 obtains theimage information of the second image for evaluation from the imageinformation of the first image.

To classify the image information pairs into the learning data and thenon-learning data, in each of the color regions, the image informationpairs are divided at a certain numeric ratio. For example, the ratio ofthe learning data and the non-learning data is set at 4:1 or 9:1 inadvance, and the image information pairs are randomly divided into thelearning data and the non-learning data in accordance with the ratio.

In this case, a user is not required to input the image information ofthe second image, and therefore, a burden of the user is reduced.

Next, operations of the image processing device 10 will be described.

First Exemplary Embodiment

In the first exemplary embodiment, as a first example of a minimumconfiguration, operations of the image processing device 10 displayingthe screen shown in FIG. 7 will be described.

FIG. 14 is a flowchart illustrating operations of the image processingdevice 10 in the first exemplary embodiment.

To begin with, the first image obtaining unit 11 obtains imageinformation of an original image before color adjustment and imageinformation after color adjustment as a first image (step 101: a firstimage obtaining process and an acceptance process).

Next, the region determination unit 121 of the image informationobtaining unit 12 determines, of any one of the image before coloradjustment and the image after color adjustment, a region from whichimage information is to be extracted (step 102: an extraction regiondetermining process).

Then, the image information extraction unit 122 extracts imageinformation items from within a region of one of the image before coloradjustment and the image after color adjustment, which is determined bythe region determination unit 121, and from within a region of the otherimage corresponding thereto (step 103: an image information extractionprocess).

Further, the image information memory 123 stores the extracted imageinformation pairs (step 104: an image information storage process).

Note that steps 102 to 104 can be grasped as an image informationobtaining process that obtains, as to the first image, the imageinformation before color conversion and the image information aftercolor conversion corresponding thereto as an image information pair.Here, a case without the process of step 102 can be considered. Whenstep 102 is not included, image information is extracted from the entireimage.

Next, the accuracy derivation unit 124 calculates the accuracy withrespect to the extracted plural pairs of image information (step 105: anaccuracy derivation process and an accuracy output process). In otherwords, as described in FIG. 5A, the accuracy is derived from angles orcolor differences of the ending points between plural color conversionvectors. At this time, the accuracy derivation unit 124 calculates theaccuracy for each of the color regions shown in FIG. 7. Moreover, asdescribed above, at this time, the image information obtaining unit 12can also exclude a color conversion vector having an angle or a colordifference significantly different from those of other color conversionvectors.

Further, the second image obtaining unit 14 obtains image information ofthe second image for evaluation (step 106: a second image obtainingprocess and the acceptance process).

Then, the second accuracy evaluation display 15 creates displayinformation to display the calculated accuracy (step 107: a displayinformation creation process). Specifically, the second accuracyevaluation display 15 creates display information to display the screenas shown in FIG. 7.

Further, the second accuracy evaluation display 15 outputs the createddisplay information to the display device 20 (step 108: a displayinformation output process and a display control process).

As a result, on the display 22 of the display device 20, the screenshown in FIG. 7 is displayed.

Second Exemplary Embodiment

In the second exemplary embodiment, as a second example of the minimumconfiguration, operations of the image processing device 10 in themodified example will be described.

FIG. 15 is a flowchart illustrating the operations of the imageprocessing device 10 in the second exemplary embodiment.

In FIG. 15, since steps 201 to 203 are similar to steps 101 to 103,descriptions thereof will be omitted.

After step 203, the classification unit 125 classifies the imageinformation pairs extracted in the image information extraction unit 122into the learning data and the non-learning data (step 204).

Then, the image information memory 123 stores the learning data (step205: a learning data storage process).

Note that steps 202 to 205 can be grasped as an image informationobtaining process that obtains, as to the first image, the imageinformation before color conversion and the image information aftercolor conversion corresponding thereto as an image information pair.

Next, the accuracy derivation unit 124 calculates the accuracy withrespect to the extracted plural pairs of image information (step 206:the accuracy derivation process and the accuracy output process).

Then, the second image obtaining unit 14 obtains the non-learning dataas image information of the second image for evaluation (step 207: asecond image obtaining process and the acceptance process).

Since the following steps 208 to 209 are similar to steps 107 to 108,descriptions thereof will be omitted.

Third Exemplary Embodiment

In the third exemplary embodiment, a description will be given of a casein which, in addition to the first exemplary embodiment, the user'sevaluation is accepted and the image information of the third image,which is the additional image, is obtained to create the conversionrelationship.

FIG. 16 is a flowchart illustrating the operations of the imageprocessing device 10 in the third exemplary embodiment.

In FIG. 16, since steps 301 to 308 are similar to steps 101 to 108,descriptions thereof will be omitted.

After step 308, the user looked at the screen in FIG. 7 inputsevaluation to, for example, the user's evaluation acceptance screenshown in FIG. 8. Then, the user's evaluation acceptance unit 16 acceptsthe inputted user's evaluation (step 309: a user's evaluation acceptanceprocess). Here, it is assumed that user's evaluation is provided in twolevels of “good” or “bad”.

Next, the user's evaluation acceptance unit 16 determines whether or notthe user's evaluation is “good” (step 310: evaluation determinationprocess).

As a result, when the user's evaluation is “good” (Yes in step 310), theadditional information deriving unit 17 determines whether or not theobtained image information pairs are insufficient (step 311:insufficiency determination process).

When the image information pairs are not insufficient (No in step 311),the color conversion coefficient calculation unit 19 creates the colorconversion model (step 312: a color conversion model creation processand a color conversion property creation process). Still further, thecolor conversion coefficient calculation unit 19 creates the conversionrelationship, such as the three-dimensional LUT, based on the colorconversion model (step 313: a conversion relationship creation process).

In other words, in the exemplary embodiment, when the user's evaluationaccepted by the user's evaluation acceptance unit 16 exceeds apredetermined criterion (in this case, user's evaluation is “good”) andthe image information pairs suffice the required number, the colorconversion model is created.

In contrast thereto, when there is insufficiency in step 311 (Yes instep 311), the process proceeds to step 317.

Moreover, when the user's evaluation is “bad” in step 310 (No in step310), the additional information deriving unit 17 determines whether ornot the accuracy obtained in step 305 satisfies a target (step 314: anaccuracy determination process). The additional information derivingunit 17 provides a predetermined limit value with respect to theaccuracy, and, based on the limit value, determines whether or not theaccuracy satisfies the target.

As a result, when the accuracy satisfies the target (Yes in step 314),the limit value is corrected (step 315: a limit value correctionprocess). That is, in this case, in spite of the fact that the accuracysatisfies the target, the user's evaluation is “bad”; therefore, it canbe considered that the accuracy level is low for the requirements of theuser. Consequently, the limit value is corrected toward the higheraccuracy level (to result in more precise level).

In contrast thereto, when the accuracy does not satisfy the target (Noin step 314), the additional information deriving unit 17 obtains colorregions required for the image information added by the third image, andfurther obtains the number of pairs to be added required as the thirdimage (step 316: an additional number of pairs derivation process).

Next, the additional information deriving unit 17 creates displayinformation of the additional information screen as shown in FIG. 11,and displays thereof on the display 22 (step 317: an additionalinformation display process). This prompts the user to input the thirdimage.

In other words, in the exemplary embodiment, when the user's evaluationaccepted by the user's evaluation acceptance unit 16 is not more thanthe predetermined criterion (in this case, user's evaluation is “bad”),the image information of the third image is obtained.

Moreover, even if the user's evaluation accepted by the user'sevaluation acceptance unit 16 exceeds the predetermined criterion (inthis case, user's evaluation is “good”), when the image informationpairs do not suffice the required number (in this case, No in step 311),the image information of the third image is obtained.

When the third image is inputted by the user, the third image obtainingunit 18 obtains image information of the third image (step 318: a thirdimage obtaining process and the acceptance process).

Further, the image information obtaining unit 12 selects an image to beused from the first image obtained in step 301 and the third imageobtained in step 318 (step 319: an image selection process). Thereafter,the process proceeds to step 302.

According to the first exemplary embodiment and the second exemplaryembodiment as described above, the accuracy of plural pairs of imageinformation as to the first image inputted by the user for each colorregion is displayed, and thereby, it is possible to quantitativelyevaluate the color conversion model created based on the first image.Moreover, a user who observed the evaluation can determine whether ornot the additional images are needed. Moreover, the user can determinefor which color region the additional images are needed. In other words,when the accuracy is low, it is possible to recognize where the problemis.

Moreover, according to the third exemplary embodiment, in additionthereto, a user who observed the accuracy information providesevaluation, and, in response to the result, it is possible to determinewhether or not an additional third image is necessary on the imageprocessing device 10 side. Moreover, on this occasion, color regions oradditional number of pairs required as the third image can be presentedto the user. Moreover, by observing the accuracy after the third imageis added, it becomes easier for the user to recognize which image to beinputted has a problem. In other words, when the accuracy is notimproved even though the number of images is increased, it can beunderstood that many images having different directionality in coloradjustment are included. Moreover, when the accuracy of the specificcolor region is not improved even though the number of images isincreased, it can be understood that the images containing the color ofthis color region are insufficient.

This makes it possible to secure the images necessary in each colorregion and to create the highly accurate color conversion model.Further, the conversion relationship created from the color conversionmodel also has high accuracy.

Note that, in the first exemplary embodiment to the third exemplaryembodiment, description is given of the case in which the screen shownin FIG. 7 is displayed; however, it is possible to display the screenshown in FIG. 6, 8 or 9. In this case, also, upon observing theaccuracy, the user can determine whether or not addition of images isnecessary, or input evaluation.

Moreover, in the above-described examples, the image before coloradjustment is an image photographed by the camera 40; however, the imageis not limited thereto, and any image can be adopted.

Further, in FIG. 6 or 7, the portions of “After” may not be displayeduntil the third image is added. Moreover, it may be possible to displaythese portions in gray until the third image is added, and after thethird image is added, the portions are updated to be changed into thenormal display.

Note that the above-described processing performed by the imageprocessing device 10 can be grasped as an image processing method. Inother words, it can be considered that the processing performed by theimage processing device 10 is an image processing method including atleast two processes of the following (I) and (II).

(I) An image information obtaining process that obtains, as to the firstimage composed of an image before color conversion and an image aftercolor conversion, image information before color conversion and imageinformation after color conversion corresponding thereto as an imageinformation pair.

(II) A display information creation process that creates displayinformation displaying accuracy of plural pairs of image information forconverting image information of an image of the first image before colorconversion into image information of an image after color conversion,the plural pairs of image information being obtained by the imageinformation obtaining process.

Moreover, the above-described processing performed by the display device20 can be grasped as an image displaying method. In other words, it canbe considered that the processing performed by the display device 20 isan image displaying method including at least two processes of thefollowing (III) and (IV).

(III) A display information obtaining process that obtains, when a colorconversion property converting image information of an image beforecolor conversion into image information of an image after colorconversion is created, as to the first image composed of an image beforecolor conversion and an image after color conversion, displayinformation for displaying accuracy of image information pairs, eachbeing composed of image information before color conversion and imageinformation after color conversion corresponding thereto for each colorregion.

(IV) A display process that displays accuracy based on theabove-described display information.

<Description of Program>

The processing performed by the image processing device 10 in theexemplary embodiments described above is provided as, for example, aprogram such as application software.

Consequently, the processing performed by the image processing device 10in the exemplary embodiments can be grasped as a program causing acomputer to implement: an acceptance function that accepts imageinformation pairs, each being composed of image information before colorconversion and image information after color conversion; an accuracyoutput function that outputs accuracy of a color conversion propertyfrom plural image information pairs accepted by the acceptance function;and a display control function that, when the acceptance functionaccepts new image information pairs, controls to display, on a displaydevice, at least image information created by color conversion of imageinformation of the new image information pairs before color conversionbased on the color conversion property created by the color conversionproperty creation function from the image information pairs havingalready been accepted by the acceptance function and image informationof the new image information pairs after color conversion.

Moreover, the processing can be grasped as a program causing a computerto implement: an acceptance function that accepts image informationpairs, each being composed of image information before color conversionand image information after color conversion; an accuracy outputfunction that outputs accuracy of a color conversion property fromplural image information pairs accepted by the acceptance function; acolor conversion property creation function that creates a colorconversion property from the plural image information pairs accepted bythe acceptance function; and a display control function that controls todisplay the accuracy outputted by the accuracy output function on adisplay device for each color region.

Note that it is possible to provide the program that implements theexemplary embodiment by a communication tool, of course, and it is alsopossible to store thereof in a storage medium, such as a CD-ROM, to beprovided.

The foregoing description of the present exemplary embodiments of thepresent invention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Thepresent exemplary embodiments were chosen and described in order to bestexplain the principles of the invention and its practical applications,thereby enabling others skilled in the art to understand the inventionfor various embodiments and with the various modifications as are suitedto the particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An image processing device comprising: anacceptance unit that accepts an image information pair composed of imageinformation before color conversion and image information after colorconversion; an accuracy output unit that outputs accuracy of a colorconversion property from a plurality of the image information pairsaccepted by the acceptance unit; a color conversion property creationunit that creates the color conversion property from a plurality of theimage information pairs accepted by the acceptance unit; and a displaycontrol unit that, when the acceptance unit accepts a new imageinformation pair, controls to display, on a display device, at leastimage information created by color conversion of image information ofthe new image information pair before color conversion based on thecolor conversion property created by the color conversion propertycreation unit from the image information pair that has already beenaccepted by the acceptance unit and image information of the new imageinformation pair after color conversion.
 2. The image processing deviceaccording to claim 1, wherein the display control unit controls todisplay, on the display device, the accuracy of the color conversionproperty before and after the new image information pair is added. 3.The image processing device according to claim 1, wherein the displaycontrol unit controls to display, on the display device, a user'sevaluation acceptance screen that accepts user's evaluation of colorconversion of image information of the image information pair beforecolor conversion.
 4. The image processing device according to claim 3,wherein, when the user's evaluation accepted by the user's evaluationacceptance screen is not higher than a predetermined criterion, thedisplay control unit controls to display, on the display device, animage for further accepting an image information pair.
 5. The imageprocessing device according to claim 4, wherein the display control unitcontrols to display, on the display device, a color region required toan image information pair to be further accepted.
 6. The imageprocessing device according to claim 4, wherein the display control unitcontrols to display, on the display device, the number of pairs to beadded that is required as an image information pair to be furtheraccepted.
 7. The image processing device according to claim 1, whereinthe display control unit controls to display, on the display device, asufficiency level of the image information pair required to create thecolor conversion property.
 8. The image processing device according toclaim 7, wherein the display control unit controls to display, on thedisplay device, the sufficiency level of the image information pairbefore and after the image information pair is obtained.
 9. An imageprocessing system comprising: a photographing device that takes aphotograph of a photographing subject; and an image processing devicethat applies color conversion to an image taken by the photographingdevice, wherein the image processing device comprises: an acceptanceunit that accepts an image information pair composed of imageinformation before color conversion and image information after colorconversion; an accuracy output unit that outputs accuracy of a colorconversion property from a plurality of the image information pairsaccepted by the acceptance unit; a color conversion property creationunit that creates the color conversion property from the plurality ofthe image information pairs accepted by the acceptance unit; and adisplay control unit that, when the acceptance unit accepts a new imageinformation pair, controls to display, on a display device, at leastimage information created by color conversion of image information ofthe new image information pair before color conversion based on a colorconversion property created by the color conversion property creationunit from the image information pair that has already been accepted bythe acceptance unit and image information of the new image informationpair after color conversion.
 10. A non-transitory computer readablemedium storing a program that causes a computer to execute functions,the functions comprising: accepting an image information pair composedof image information before color conversion and image information aftercolor conversion; outputting accuracy of a color conversion propertyfrom a plurality of the accepted image information pairs; creating thecolor conversion property from the plurality of the accepted imageinformation pairs; and, when a new image information pair is accepted,controlling to display, on a display device, at least image informationcreated by color conversion of image information of the new imageinformation pair before color conversion based on the color conversionproperty created from the image information pair that has already beenaccepted and image information of the new image information pair aftercolor conversion.